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Abstract 


This  research  addresses  the  implementation  of  an  electronic  element, 
which  emulates  the  biological  synaptic  interconnection,  in  an  artificial 
electronic  neural  system.  The  basic  interconnection,  or  the  weight,  consists 
of  an  electrically  reprogrammable,  nonvolatile,  analog  conductance  which 
programs  at  5V  levels.  In  addition,  the  fabrication  technology  for  this 
synaptic  interconnection  is  compatible  with  existing  CMOS  VLSI  process. 
The  attractive  features  of  this  synaptic  weight  will  be  discussed  in  this 
report.  Furthermore,  this  report  examines  the  material  needs,  the  device 
structures,  the  use  of  the  synaptic  weights  in  a  two-tap  weight  linear 
adaptive  neural-like  circuit  and  the  issue  of  integrating  both  the  synaptic 
weight  elements  and  the  peripheral  circuit  onto  a  single  silicon  wafer. 


1.  Introduction 

The  current  surge  of  enthusiasm  for  neural  network  aims  to  construct  systems  that  can  leam 
or  modify  their  behavior  according  to  the  environment.  There  are  many  similarities  which  exist 
between  this  new  class  of  machine  and  human  beings.  One  of  these  similarities  is  the  massive 
parallelism  in  processing  information.  Parallel  processing1  concepts  are  in  stark  contrast  to  the 
operations  of  modern  digital  computers  that  perform  large  numbers  of  sequential  operations  very 
rapidly  and  accurately. 

Researchers  believe  the  synaptic  junctions  in  a  neural  system  are  the  local  memory  sites 
and  provide  the  physiological  basis  for  the  distributed  parallel  systems.2,  3  These  synapses  are  not 
only  modifiable  but  also  serve  the  functions  of  storing  and  transmitting  information  from  neuron  to 
neuron.  To  reduce  the  complex  modelling  required  for  the  synaptic  interconnection,  the 
representation  of  the  synapse  has  been  simplified  to  a  single  ideal  junction  between  the  output  of 
neurons  (axons)  and  the  inputs  to  neurons  (dendrites).  Synaptic  modification  requires  information 
from  the  input  and  the  output  of  the  neuron  in  order  to  perform  complex  recognition.  Therefore,  the 
nature  of  the  synaptic  junction  and  the  principle  or  algorithm  which  controls  local  organization  at 
the  neuron  level  become  two  central  issues  pertaining  to  neural  networks  research. 

The  recent  interest  in  neural  networks4, 3  is  a  direct  consequence  of  the  programmability 
which  is  an  essential  feature  of  learning  machines,  associative  memories,  and  adaptive  signal 
processors.  Programmability  requires  a  modification  of  the  synaptic  strength  in  the  language  of 
neurobiology.  If  we  seek  an  efficient  hardware  implementation  of  electronic  neural  systems,  then 
the  synapses  -  as  well  as  the  network  itself-  should  be  analog.  Several  attempts  have  been  made  to 
realize  programmable  synapses,  either  digitally6  or  with  temporary  storage  on  the  input  capacitance 
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of  a  MOS  Transistor7, 8  to  alter  the  latter’s  analog  conductance.  The  former  approach  stores  the 
weight  information  in  digital  registers  and  thus  suffers  from  excessive  chip  area  and  power 
consumption.  On  the  other  hand,  although  the  MOS  Transistor  provides  an  analog  synaptic 
strength  f  weight)  in  a  small  chip  area,  the  weight  is  temporary  and  requires  periodic  refresh  similar 
to  a  DRAM.  Thus,  this  dynamic  refresh  approach  lacks  the  nonvolatility  and  storage  properties  of 
an  EEPROM  cell.  Researchers  at  Intel  have  reported  an  electrically  trainable  artificial  neural 
network  -with  floating  gate  device  as  the  synaptic  element.9  Although  floating  gate  device  has  the 
property  of  nonvolatility,  its  high  programming  voltage  requirement  prevents  it  from  being 
technologically  compatible  with  scaled  CMOS  process. 

In  this  research  report  we  describe  a  new  approach  to  obtain  an  electrically  reprogrammable  or 
modifiable  synaptic  weight  to  be  used  as  a  basic  functional  element  in  electronic  neural  systems. 
The  salient  features  of  this  network  element  are  the  following: 

•  Low  programming  voItages(o-lOV)  which  are  compatible  with  peripheral 
CMOS  VLSI  technology  in  contrast  with  Floating  Gate  approaches. 

•  Low  power  dissipation  ( <  1  uW). 

•  Dynamic  Range  of  1000:1  (60  dB). 

•  Nonvolatile  features  which  mimic  biological  synapses  with  respect  to  memory 
loss  (e.g.  20*®  of  the  information  available  after  10  years)  and  reinforced 
learning  (e.g.  successive  interrogation  enhances  memory  retention). 

•  Small  synaptic  area  on  a  VLSI  chip  (e.g.  less  then  20um2  for  1.25  |im  feature 
sizes). 

•  Extensive  erase/write  programming  cycles  are  possible  with  this  synapse  ( > 

108  cycles)  in  contrast  with  Floating  Gate  approaches. 

•  Inherent  radiation  damage  resistance  beyond  a  total  dosage  of  IMRad  (Co30) 
and  lflP  Rad/sec  transient  which  is  not  possible  with  Floating  Gate  technology. 

Thus,  if  radiation  damage  resistance  of  neural  networks  is  an  important  issue, 
then  the  SONOS  devices  have  demonstrated  success  in  this  area. 

The  basic  nonvolatile  derice  structure,  which  we  describe  in  this  report  was  first  introduced  as 
a  digital  nonvolatile  memory  ceil  in  the  summer  of  1987  at  the  IEEE  Device  Research  Conrerence*0 
by  researchers  at  Lehigh  University.  We  have  had  a  continual  involvement  over  a  20  year  period 
v::h  nonvolatile  memories,  beginning  in  the  late  60’s  where  we  had  programming  voltages  oi  25  v .  :o 
the  late  50’s  with  our  novel  5V  SONOS  device  structures.  During  this  time  period  we  introduced  the 
use  of  CCD’s  and  nonvolatile  memories**'  “■  13  in  nonvolatile  charge  addressed  memories 
NO  VC  AM'.  These  ideas  have  been  employed  recently  for  neural  network  circuits  by  researchers  at 


Lincoln  Laboratories. 14  Our  recent  work  recognizes  the  inherent  analog  conductance  aspect  of  the 
nonvolatile  SONOS  memory  device  which  makes  it  a  perfect  candidate  for  the  modifiable  synapse  in 
an  electronic  neural  system. 

In  addition  to  the  realization  of  an  electronic  element  to  simulate  the  synaptic  interconnections 
of  a  neural  network,  we  must  have  a  method  or  algorithm  to  change  or  reprogram  these 
interconnections  and,  thus,  alter  the  connectivity  of  the  neural  network.  We  have  had  experience 
with  a  particular  form  of  an  algorithm,  namely,  the  Widrow-Hoff  Least  Mean  Square  (LMS)15  error 
algorithm  or  in  neural  network  terminology  -  the  so-called  ’delta  rule’.  In  the  late  70’s  we  researched 
a  CCD  Adaptive  Analog  Signal  Processor16, 17  which  realizes  the  ’delta  rule’  with  CCD  analog  delay 
lines  and  electrically  reprogrammable  MNOS  analog  conductance  weights.  These  weights  were 
nonvolatile  memory  transistors  whose  analog  conductance  was  programmed  with  voltages  ranging 
from  15-25V.  Our  recent  work  on  ’scaling’  these  programmable  analog  conductances  has  resulted  in 
a  new  device  structure,  called  the  SONOS  nonvolatile  memory  transistor,  which  can  be 
reprogrammed  with  voltages  ranging  from  5-10V.  This  work  has  recently  been  described  at  the  1991 
11th  IEEE  Nonvolatile  Semiconductor  Memory  Workshop.18  These  voltage  levels  are  compatible 
with  ’scaled’  CMOS  VLSI  technology  which  has  12- 15V  breakdown  voltages  for  1.25um  feature  sizes. 
In  this  report  we  describe  our  recent  work  on  the  electrically  reprogrammable  (modifiable)  SONOS 
nonvolatile  synapse  and  a  simple  electronic  neuron  with  2  synaptic  weights.  We  discuss  this  two-tap 
weight  linear  adaptive  neuron  in  terms  of  the  technology,  the  electrical  characteristics  of  the 
synapses,  and  their  performance  in  this  simple  test  vehicle  -  a  'delta  rule’  adaptive  signal  processor. 

2.  Technology  and  Characterization  of  SONOS  Synaptic  Weight 

The  programmable  synapse  is  the  result  of  an  ongoing  effort  at  Lehigh  University  to  scale'  the 
programming  voltages  required  to  alter  the  analog  conductance  of  a  nonvolatile  me.nory  transistor 
with  a  multi-layer  i  oxide-nitride-oxide)  gate  insulator  as  shown  in  Fig.  1.  Recent  efforts  in  scaling 
this  device  have  resulted  in  a  SONOS  (SiiicoruBlocking  Oxide/Nitride/Tunneling  OxddaSiiicon) 
nonvolatile  memory  transistor  which  is  electrically  reprogrammable  at  CMOS  voltage  levels. 
Typically,  the  tunneling  oxide  is  15-25A.  ,  the  storage  nitride  is  50-100A  and  the  blocking  oxide  is 
3c-30A  .  Fig.2  shows  the  Transmission  Electron  Microscope  >TEM)  photograph  of  the  cross  sectionai 
view  of  the  SONOS  transistor.  This  device  is  similar  to  a  SNOS  transistor  except  for  the  addition  of 
the  blocking  oxide  which  is  used  to  inhibit  injection  of  carriers  from  the  poivsilicon  gate  electrode 


and  also  to  improve  the  memory  retention  by  prohibiting  the  transfer  of  stored  charge  from  the 
nitride  to  the  gate  electrode.  As  the  result,  the  blocking  oxide  permits  the  entire  dielectric  sandwich 
to  be  scaled  to  dimensions  where  the  programming  voltages  ranging  from  5-10  V  are  possible. 


When  the  SONOS  device  is  subjected  to  a  positive  (or  negative)  programming  pulse,  electrons 
(or  holes)  are  injected  into  the  silicon  nitride  layer  by  means  of  tunneling  across  the  thin  tunnel 
oxide.  The  injected  charges  are  trapped  by  the  silicon  nitride  and  thus  shift  the  threshold  voltage 
positively  ("or  negatively).  The  threshold  voltage  of  a  SONOS  transistor  can  be  written  as 
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where  is  the  bulk  potential,  bog  is  the  gate  to  semiconductor  workfunction,  Qf  is  the  fixed  charge 
at  the  tunneling  oxide-silicon  interface,  eQX  and  are  the  dielectric  permittivities  of  the  oxide  and 
nitride,  -3-  is  the  dielectric  permittivity  of  the  bulk  silicon,  xot  is  the  tunnel  oxide  thickness,  x.D  is  the 
blocking  oxide  thickness,  xn  is  the  nitride  thickness,  x  is  the  charge  centroid  in  the  insulator,  and  QN 
is  the  charge  stored  in  the  nitride,  N3  is  the  bulk  doping  density,  and 
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We  assume  that  the  tunnel  oxide  and  blocking  oxide  have  the  same  dielectric  permittivity;  even 
though,  it  is  known  that  the  tunnel  oxide  is  silicon  rich  and  the  blocking  oxide  is  an  oxynitride.  The 
values  of  the  charge  centroid  X  and  the  charge  stored  in  the  nitride  QN  will  change  as  the  device  is 
written  or  erased.  The  analog  conductance  of  the  SONOS  synaptic  weight  may  be  written  as 
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where  u3ir  is  the  effective  carrier  mobility,  Vqq  is  the  read  voltage,  and  is  the  electrically 
modifiable  threshold  voltage  given  in  equation  <  1).  Therefore,  there  are  two  ways  which  the  analog 
channel  conductance  can  be  altered:  1:  change  the  value  of  V^g  or  (2)  change  the  value  of  V-y.  In 
our  study,  the  latter  approach  is  chosen. 

The  SONOS  transistors  are  characterized  for  their  memory  properties  by  using  the  test  station 
described  in  Anirban  Roy's  Masters  Thesis.-9  This  test  station  allows  one  to  take  both  the 
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erase/write  and  retention  measurements.  To  investigate  the  memory  loss/rentention  properties  of 
the  synaptic  weight  element,  retention  measurements  are  taken.  The  retention  characteristics  are 
obtained  by  applying  positive  (negative;  five  volts  to  the  gate  for  10  seconds  to  place  the  device  in  the 
■write  (erase)  state  and  then  measuring  the  turn-on  voltage  after  a  varying  delay  time.  The  turn-on 
voltage  is  related  to  the  threshold  voltage  by  ^ 

21nc 

VT  =  +  (4) 

with  IDS  as  the  forced  drain  to  source  current  during  measurement  and 

P  =  CelT  ' 

where  W  is  the  width  of  the  transistor,  L  is  the  length  of  the  transistor,  and  ueiy  is  the  effective 
mobility.  The  effective  mobility  is  the  bulk  mobility  reduced  by  Coulombic  and  surface  scattering  of 
carriers  in  the  inversion  layer.  This  mobility  is  influenced  by  the  gate  and  substrate  voltages.-0  For 
a  SONOS  transistor  retention  measurements  indicate  that  greater  than  20  percent  of  the  memory 
window  remains  after  a  projected  10  year  delay  time  as  shown  in  Fig.  3.  The  erase/write 
measurements  indicate  the  programming  speed  of  the  synaptic  weight  element.  To  measure  the 
■writing  (erasing;  speed,  negative  (positive)  five  volts  are  applied  to  the  gate  for  10  seconds  to  place 
the  device  in  the  erase  (write)  state.  Then,  positive  (negative)  five  volts  are  applied  to  the  gate  with 
varying  pulse  widths  and  the  tum-on  voltage  is  measured  after  each  pulse  width.  The  erase/write 
characteristics  of  the  SONOS  memory  transistor  are  shown  in  Fig.  4.  A  wide  dynamic  range  is  one 
of  the  essential  properties  for  the  synaptic  weight  element,  and  Fig.  5  illustrates  a  60  dB  in  dynamic 
range  after  r5V  programming  for  the  SONOS  synaptic  weight.  In  addition,  a  recent  study  in 
reliability  has  demonstrated  the  inherent  resistance  of  the  SONOS  memory  transistor  to  radiation 
damage  'SV^  =  O.lV,  with  V^g  =  -r  5V  at  IMRad  Co50  radiation).-1 

3.  Single-level  Linear  Adaptive  Neuron 

We  have  incorporated  the  SONOS  synaptic  weights  into  a  single-level  linear  neuron-like 
circuit  using  a  Widrow-Hoffs  delta  learning  rule.15  The  circuit  is  built  with  a  hybrid  breadboard  of 
CMOS  components  for  the  control  logic  and  the  algorithm  implementation  and  the  SONOS 
nonvolatile  memory  transistors  to  demonstrate  the  voltage  level  compatibility  of  both  SONOS  and 
CMOS  technologies.  Many  researchers  believe  that  the  neural  system  is  made  up  of  several  layers' 


of  neurons  and  Fig.  6  shows  the  multi-layer  architecture  of  an  artificial  neural  network.  The  first 
layer  of  neurons,  the  input  layer,  can  be  best  thought  as  the  sensory  neurons  in  a  human  body.  The 
weight  connections  between  the  input  layer  and  the  middle  hidden  layer  are  normally  considered  to 
be  feedforward  and  fixed.  On  the  other  hand,  the  weight  connections  between  the  middle  hidden 
layer  and  the  output  layer  are  considered  to  be  feedback  in  nature.  Our  work  has  concentrated  on 
the  implementation  of  two  neurons  in  the  hidden  layer  and  one  output  neuron  as  highlighted  in  the 
figure. 

Fig.  7  shows  the  block  diagram  of  the  single-level  linear  adaptive  neuron.  A  desired  response 
for  external  teacher),  dim),  is  presented  to  the  neuron  as  the  training  signal.  If  the  output  of  the 
linear  adaptive  neuron  is  not  trained,  then  there  exists  a  mismatch  between  the  output  of  the  linear 
adaptive  neuron,  yim),  and  the  desired  response,  dim). 

zim )  =  dim)  -  y(m i  (6) 

where  Zimi  is  the  error  g<merated.  This  error  is  then  used  by  a  learning  algorithm,  namely  the 
Clipped-data  Least  Mean  Error  algorithm,  to  minimize  the  error  generated  and  thereby  training  the 
neuron  to  the  correct  response.  This  single-level  linear  adaptive  neuron  has  two  tap  weights,  each 
weight  is  comnosed  of  two  SONOS  analog  electrically  reprogrammable  conductances  as  shown  in 
Fig.  5.  Since  the  synaptic  weight  must  be  either  positive  or  negative  in  value,  we  have  chosen  a 
differential  weighting  scheme.  If  the  analog  conductance  connecting  the  positive  summing  path  to 
the  differential  operational  amplifier  is  greater  than  the  analog  conductance  connecting  the  negative 
summing  path  to  the  differential  operational  amplifier,  then  the  weight  is  positive  in  value.  On  the 
other  hand,  if  the  opposite  case  is  true,  then  the  weight  is  negative  in  value.  Positive  weight  value 
corresponds  to  the  excitatory  synaptic  strength  and  the  negative  weight  value  corresponds  to  the 
inhibitory  synaptic  strength. 

In  operation,  the  input  signal  ,tu)  is  passed  through  a  switched  capacitor  analog  delay  line 
where  the  input  signal  is  sampled  and  delayed  to  create  two  tapped  signal  outputs  x0im)  and  x,(m). 
These  tapped  signals  multiply  to  their  corresponding  programmable  weights  VF0  and  Wj  and  the 
result  is  summed  linearly  at  the  summing  amplifier.  The  output  y\m)  can  be  expressed  as: 

}'( m  I  =  V  i~ ) 

c=U 
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where  m  is  the  time  index  and  k  is  the  spatial  index.  A  correlated  double  sampling  technique22is 
employed  in  the  circuit  to  remove  the  unwanted  noise  and  offset  voltages  introduced  by  the  summing 
amplifiers.  The  linear  adaptive  neuron  is  configured  to  perform  Widrow-HofPs  delta  rule  as: 

Wk{m+ 1)  =  Wk(m)  +  A  Wk(m)  (8) 

where  A  W(m>  is  the  incremental  weight  to  be  calculated  by  the  clipped-data  least  mean  square  error 
fC-LMSE)  algorithm23: 

A  Wk(m)  =  2u  !e(m)|  Sgn{e(m)]  Sgn[.x(m-k)}  (9) 

where  u  is  the  convergence  factor.  Compared  to  the  regular  Least  Mean  Square  Error  algorithm,  the 
input  signal  ampiitude  is  clipped  in  the  learning  algorithm.  This  algorithm  eliminates  the  usage  of  a 
four  quadrant  multiplier  needed  for  the  LMS  error  algorithm.  The  sign  multiplication  in  the 
incremental  weight  calculation  is  essentially  an  Exclusive  OR  operation  and  the  output  of  the 
Exclusive  OR  gate  controls  the  path  of  proper  gate  programming  voltage  for  the  SONOS  synaptic 
weight.  If  the  convergence  factor  is  small,  then  the  system  will  minimize  the  misadjustment  caused 
by  the  variance  of  the  weights;  however,  this  also  results  in  a  long  convergence  time.  Conversely,  if 
we  choose  to  use  a  larger  convergence  factor,  then  the  convergence  time  of  the  system  is  shortened 
■with  the  penalty  of  larger  misadjustment.  The  backpropagating  error  is  used  to  calculate  the 
adjustments  to  minimize  the  system  error  as  shown  in  equation  (9).  Once  the  error  is  minimized, 
the  system  is  said  to  be  in  its  steady  state  condition24  where  the  output  of  the  system,  y(mj,  is  the 
best  match  of  the  training  signal,  d(m),  or  the  'external  teacher'. 

The  incremental  weight  update  is  essentially  a  cross  correlation  between  the  error  and  the 
clipped  input  data  vectors.  The  update  stops  when  the  two  vectors  become  orthogonal.  Sometimes, 
the  network  may  be  overcorrected  initially,  however,  the  error  will  be  quickly  minimized  by  the 
learning  algorithm  and  the  system  reaches  its  desired  response.  The  digital  delay  line  provides  the 
sign  information  of  the  input  to  the  learning  algorithm.  A  special  steering  network  is  designed  to 
switch  the  proper  programming  voltages  to  the  gate  terminals  of  the  SONOS  transistors  once  the 


•incremental  weights  are  calculated. 


4.  Experimental  Results 

There  are  two  main  types  of  characteristics  from  which  the  electrical  performance  of  the  linear 
adaptive  neuron  can  be  evaluated.  The  first  characteristic,  namely  the  output  and  training  signals 
versus  time  characteristics,  gives  the  information  on  how  well  the  output  signal  approximates  the 
training  signal  especially  in  the  phase  relationship  between  these  two  signals.  The  second 
characteristic,  namely  the  error  signal  versus  time  characteristics,  shows  how  fast  the  linear 
adaptive  neuron  adapts  before  it  reaches  its  minimum  error.  A  typical  output  and  training  signals 
versus  time  characteristic  consists  of  two  parts:  the  initialized  and  the  adapted  part.  In  the 
initialized  part,  the  weights  are  first  initialized  to  a  known  state  (either  the  fully  positive  or  the  fully 
negative  state)  and  then  the  weights  are  subjected  to  a  reading  voltage  to  read  out  the  weight 
information  and  the  output  signal  and  the  training  signal  are  compared  and  recorded.  The  linear 
adaptive  neuron  is  then  allowed  to  adapt  itself  to  the  training  signal  and  the  results  are  shown  in 
the  adapted  part  of  the  characteristics.  Figure  9  shows  the  output  and  training  signal  versus  time 
characteristic. 

A  typical  error  signal  versus  time  characteristic  is  obtained  'with  initialized  weight  values  and 
monitoring  the  error  signal  with  time.  Our  observation  indicates  the  weight  initialization  scheme 
affects  the  convergence  behavior  of  the  linear  adaptive  neuron.  This  phenomenon  is  attributed  to 
the  nonsymmetric  erase  and  write  characteristics  of  the  SONOS  transistor.  Therefore,  one  weight 
initialization  scheme  may  require  more  erase  action  taking  place  than  another  weight  initialization 
scheme,  causing  a  difference  in  convergence  characteristics.  Figures  10  shows  a  typical  error  versus 
time  characteristic. 

5.  Technical  Progress 

During  the  research  period  from  October  1990  to  March  1991.  several  achievements  have  been 
made  under  the  contract.  First  of  ail,  a  graduate  student  supported  by  the  contract  has  received  his 
Master’s  degree  in  Electrical  Engineering  in  October  1990  and  part  of  his  thesis  was  presented  in  the 
1991  IEEE  Nonvolatile  Semiconductor  Memory  Workshop  in  February  1991.  In  addition,  our 
research  results  'will  be  presented  at  the  Electro  International  Conference  in  New  York  as  well  as  an 
invited  presentation  at  the  Wright  Patterson  .Aar  Force  Base  in  Dayton.  Ohio  during  the  month  or 
April. 


The  course  of  optimizing  the  synaptic  weight  element  has  resulted  in  a  new  fabrication  run 
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aiming  to  produce  faster  devices  to  improve  the  linear  adaptive  neuron  performance.  The 
erase/write  characterization  of  the  newly  made  devices  indicate  a  roughly  one  order  of  magnitude 
improvement  in  the  programming  speed  (determined  by  the  cross-over  time  as  mentioned  in  the 
previous  section)  over  the  older  devices.  Figure  11  shows  the  difference  in  programming  speed 
between  the  newly  made  devices  and  the  old  devices.  We  have  also  investigated  the  programming 
voltage  dependence  on  the  programming  speed  for  the  newly  made  devices.  Figure  11  shows  an 
improvement  of  one  order  of  magnitude  in  programming  speed  with  each  one  volt  increment  in  the 
programming  voltage.  In  order  to  further  aid  the  investigation  in  the  synaptic  weight  element 
characterization,  a  fully  computer  controlled  automatic  data  acquisition  system  specially  designed 
for  the  synaptic  weight  elements  is  needed.  This  system  is  a  natural  extension  of  the  test  station 
currently  used  in  our  studies19  and  a  proposed  block  diagram  of  the  automatic  data  acquisition  is 
shown  in  figure  12.  In  the  hardware  implementation  of  the  linear  adaptive  neuron,  a 
reprogrammable  controlling  clock  generator  was  built.  This  clock  generator  can  supply  up  to  15 
different  clocking  waveforms  'with  the  clocking  patterns  pre-programmed  in  an  EPROM  ceil.  In 
addition,  this  generator  can  house  several  clocking  schemes  which  can  be  selected  by  changing  a  DIP 
switch  setting. 

Integration  of  the  linear  adaptive  neuron  onto  a  single  silicon  wafer  is  one  of  the  main  goals  of 
our  research  efforts.  We  have  acquired  a  computer  aided  design  software  package,  developed  by  the 
Mentor  Graphics  Corporation,  and  implemented  on  our  SUN  workstation.  In  order  to  make  full  use 
of  the  newly  acquired  software,  we  have  developed  a  technology  file  which  describes  the  fabrication 
capability  of  our  microelectronics  laboratory.  We  have  also  started  laying  out  small  sample  building 
blocks  to  become  familiar  with  the  methodology  of  the  system.  One  of  the  most  important  building 
blocks  is  the  design  of  the  operational  amplifier  which  is  used  in  the  analog  delay  line  chain  and  the 
discrete  analog  signal  processing  of  the  neuron  output  signal.  SPICE  simulations  have  been 
performed  for  a  number  of  different  designs  for  possible  implementation  in  the  integrated  linear 
adaptive  neuron. 

6.  Proposed  Investigations 

To  take  full  advantage  of  the  SONOS  synaptic  weight  element,  more  experiments,  analysis 
and  fabrication  of  the  devices  are  required.  Besides  device  fabrication,  automation  of  device 
characterization  is  of  concern  in  the  next  phase  of  our  studies.  The  layout  of  the  integrated  linear 
adaptive  neuron  will  be  the  main  focus  of  our  efforts  in  the  next  few  months.  The  design  will  be 
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tested,  laid  out,  extracted  and  simulated  to  verify  the  functionality.  We  believe  the  integration  of  the 
signal  processing  circuitry  as  well  as  the  synaptic  weight  element  onto  a  single  silicon  wafer  will  be 
an  important  advancement  to  the  Artificial  Intelligence  Neural  Network  Technology  field. 

7.  Conclusions 

The  SONOS  nonvolatile  memory  transistor  has  been  shown  to  be  an  ideal  electronic  element 
for  the  electrically  reprogrammable  analog  conductance  in  an  artificial  neural  network.  We  have 
demonstrated  the  attractive  features  of  this  synaptic  weight  for  the  use  of  large  neural  network 
systems,  for  instance,  low  programming  voltage  (5-10V),  low  power  dissipation(<lpW  /  synapse), 
small  chip  area  ('estimated  20um2/  weight  cell  for  a  1.2  pm  feature  size),  a  dynamic  range  of  60  dB, 
good  memory  retention  (20  %  window  at  a  projected  10  years  period),  and  endurance  beyond  10‘ 
erase/write  cycles.  In  addition,  the  SONOS  synaptic  weight  has  inherent  resistance  to  radiation 
damage  (AV'(>1  =  0.1V',  with  V  =  at  IMRad  Co30  radiation).  We  have  been  continuing  our  efforts  in 
optimizing  the  modifiable  synaptic  weights  to  provide  better  electrical  characteristics  for  neural 
network  applications. 

We  have  also  incorporated  the  SONOS  synaptic  weights  into  a  single-level  two  tap  linear 
adaptive  neuron  employing  a  Widrow-Hoffs  delta  learning  rule.  The  combination  of  CMOS  control 
circuits  and  SONOS  synaptic  weights  has  demonstrated  the  feasibility  of  integrating  these  two 
technologies  onto  a  single  silicon  wafer.  The  initial  results  are  encouraging  and  promising  and 
provide  insight  and  direction  into  the  integration  of  these  two  technologies  to  realize  large  artificial 
neural  network  systems. 
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Figure  1.  Cross  Sectional  View  of  the  (a)  MNOS  (b)  SONOS  Electrically 
Modifiable  Synaptic  Weight  (c)  SONOS  Ideal  Energy-Band  Diagram 
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Figure  3.  Retention  Characteristics  of  a  Modifiable  SONOS  Synaptic  Weight 
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Figure  5.  Dynamic  Range  of  a  SONOS  Synaptic  Weight  After  Programming 
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Figure  6.  Conceptual  View  of  Multi-layer  Artificial  Neural  Network 
Architecture  Incorporating  the  Single- Level  Linear  Adaptive  Neuron 
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Figure  7.  Block  Diagram  of  a  Single-level  Linear  Neuron 
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Figure  8.  Electrical  Implementation  of  the  Synaptic  Weights 
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Figure  9.  Output  and  Training  Signals  versus  Time  Characteristics  >ai  Initialized  >b)  Adapted 
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Figure  10.  Error  Signal  versus  Time  Characteristics 
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Figure  11. Improved  Characteristics  Demonstrated  by  the  Newly 
Fabricated  SONOS  Synaptic  Weight  Elements 
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Figure  12.  Programming  Voltage  Dependence  in  the  Programming  Speed 
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Figure  13.  Proposed  Block  Diagram  of  the  Automated  Data  Acquisition  System 
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